Assignment 2 - Part A

Done by :

Class of DIT/FT/2B/11

Index

  1. Ensuring GPU Utilization is minimal
    1.1. Ensuring 0% Util
    1.2. Forcing Utils To 0% To Get A Clean Cluster
  2. Ensuring GPU is utilized in environment
    2.1. See the list of available devices
    2.2. Cloning project from Github
    2.3. Ensuring correct working directory
  3. Background Information
    3.1. About the CelebA dataset
    3.2. CelebA dataset
  4. Data Importing
    4.1. Load the libraries
    4.2. Creating hms_string class for time recording purposes
  5. Exploratory Data Analysis
    5.1. Basic dataframes insights
    5.2. Check for missing values
    5.3. Further data exploration
    5.4. Data filtering
  6. Feature Engineering
    6.1. Normalizing training/testing dataset images and reshaping
    6.2. Placing training images into tensorflow dataset object
  7. DCGAN Model Training Preparation
    7.1. Creating the generator
    7.2. Using the generator(untrained) to create a random image-to-create-a-random-image)
    7.3. Creating the discriminator
    7.4. Using the discriminator(untrained) to classify "real" and "fake" faces-to-classify-"real"-and-"fake"-faces)
    7.5. Define loss and optimizers for generator and discriminator
    7.6. Define training loop
    7.7. FID Score (Frechet Inception Distance))
    7.8. InceptionV3 Model For FID Score
  8. DCGAN Model Training
    8.1. Train DCGAN Model
  9. DCGAN Model Evaluation
    9.1. Plot generator and discriminator loss graph
    9.2. Make GIF of generated images
    9.3. FID Score (Frechet Inception Distance))
  10. References

1.Ensuring GPU Utilization is minimal

1.1 Ensuring 0% Util

Ensure that our slot give by Google is not utilized yet.

1.2. Forcing Utils To 0% To Get A Clean Cluster

Please do not use this step unless the cluster you are allocated to is quite full.

2.Ensuring GPU Is Utilized In Environment

2.1. See the list of available devices

This entire section can be omitted if users are not utilizing GPU at all.

2.2. Cloning project from github

This is to clone the entire repo from github. Please do not execute this unless you are running in a different session of google colab and that the previous dataset has been wiped off. It might take a while.

2.3. Ensuring correct working directory

We do this to check that we are doing in the right directory before proceeding.

3.Background Information

3.1. About The CelebA Dataset

3.2. CelebA Dataset

4.Data Importing

4.1. Load the libraries

Load the necessary libraries for usage in the entire project.

4.2. Creating hms_string class for time recording purposes

We create hms_string class for time recording purposes later on when we train our models, so that we can see the time elapsed.

5.Exploratory Data Analysis

5.1. Basic Dataframes Insights

Take a peek look at what is inside the respective dataframes first before we do something to it.

Analysis Summary: It is a commmon practice for data scientists to classify 1 as positive and 0 as negative, so we shall convert -1 (negative) to 0 to comply standard protocols. We also make image_id as the index.
Analysis Summary: We see that the highest correlation is between Heavy_Makeup and Wearing_Lipstick. The correlation is 0.80 which is pretty high. We can also see that there are some attribute pairs that have negative correlation.

5.2. Check for missing values

We check if there is any missing values in our attribute dataframe before proceeding on.

Analysis Summary: It seems like there isn't any missing values in the dataset. We can proceed on to do more EDA.

5.3. Further data exploration

We go on to take a further in-depth exploration of our dataset.

Analysis Summary: From here, we can see that the celebA dataset consists of 202599 entries, and that there are 40 recorded attributes.
Analysis Summary: All the image shape will be in (218, 178, 3). The reason why there is a 3 behind is because the images are colored images (RGB), and therefore the 3. It represents the channel. We will now proceed to filter the dataset to a more specific group of people so as to reduce our training dataset size to reduce computational power. We will filter the celebrities based on the various attributes explored above.
Analysis Summary: Based on all the boxplot, we can see that there are more females than males, more peole without moustach than those with moustach, and definetly, lesser balded celebrities than non-bald celebrities. We also have more recorded instance of closed mouth celebrities than those whos mouth are wide open.

5.4. Data filtering

We will now proceed on to only get the images of celebrities that are bald, have their mouth slighly opened, have mustache, and are males so that we can train our generator and discriminator for this specific group of celebrities. This reducess the training size we will be using later on and make the image looks clearer.

Analysis Summary: Now it seems that we have successfully partitioned the images in the csv file. We have around 65 dataset right now after all the filtering. The reason why we must ensure we partition properly in the csv file is because there is the respective image filename for the respective filtered celebrities. We will be using the image_id to select out the actual photos.
Analysis Summary: Now that we have managed to get the images based on the filtered filenames, we will now plot 30 random images to see how they look like.
Analysis Summary: From the plot above, we can double confirm that we have successfully gotten the filtered celebrity images. We will now try to see the max pixel value for a random image. All of the images are of the same size, so we can just randomly take one picture out for verification.

6.Feature Engineering

6.1 Normalizing training/testing dataset images and reshaping

Normalize the training/test images by dividing with 255. We also need to reshape the image to reduce computation power.

Analysis Summary: We have previously seen that our images max value is 255px and min value is 0. We want to normalize it such that it becomes within the range of 0 and 1. We will also proceed to standardize the values to a range between -1 and 1 as we will be using tanh activation function later on in our generator, which accepts a range only between -1 and 1.
Analysis Summary: We can see that after normalizing, our images have seem to become slightly darker and since we reformat them from (218, 178) to (128, 128), we can see that the images look slightly squashed, but still overall clear.

6.2 Placing training images into tensorflow dataset object.

We place the training images into tensorflow dataset

Analysis Summary: In one Stack Overflow Forum, i have realised that it is important that we ensure that our batch size must not be very big. Our buffer_size for tf.shuffle must not be also too big as it will result in abnormal output of the generated images. I have placed the hyper-link at the bottom under the reference section.

7.DCGAN Model Training Preparation

7.1. Creating The Generator

We will now create the generator model for our GAN. We will follow with the original architecture.

Analysis Summary: While adding the layers, we will insert an assert to check that the output shape from each layer is correct or not. If it isn't, there will be an error raised. We will not directly copy the node values from the original DCGAN paper, but we have to ensure that the step_size is the same as the pattern from the original paper.

7.2. Using the generator(untrained) to create a random image

Now we will use the generator, which we have yet to feed in any training dataset, to generate an image.

Analysis Summary: Here, we will create a noise vector of values from -1 to 1, using random uniform. We declare the vector shape to be 4 * 100 instead since we wanna see 4 outputs of height 100 vectors. We have to use tf.random as we are using the tensorflow API. We also declare the training to be False as we do not want the generator to be influenced first.
Analysis Summary: When running the above code, we will expect to see "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers)." message coming out. This is because matplotlib.imshow only accepts values ranging from 0 to 1. Our dataset is ranged from -1 to 1 and therefore, there is such an error, but it is not a big issue. We also can see that our generated images is black. Not to worry as our images is supposed to be in RGB. If we see a rainbow colored random image, it might hint that it is generating a Black and White image and we might be doing it wrongly.

7.3. Creating The Discriminator

We will now create the discriminator model for our GAN. It is almost similar to a standard CNN architecture with some changes.

7.4. Using the discriminator(untrained) to classify "real" and "fake" faces

Now we will use the discriminator, which we have yet to feed in any training dataset, to determine if an image is real or fake. If it is real, then the output values are positive, and negative values for fake images.

7.5. Define Loss and Optimizers for Generator and Discriminator

We will now define the loss function and optimizers for both of the models here for observations later on.

7.6. Define Training Loop

We will now define the training loop so that we can fit in our data. We will include our train_step, generate_and_save_images and train together.

Analysis Summary: The training loop begins with generator receiving a random seed as input. That seed is used to produce an image. The discriminator is then used to classify real images (drawn from the training set) and fakes images (produced by the generator). The loss is calculated for each of these models, and the gradients are used to update the generator and discriminator.

7.7. FID Score (Frechet Inception Distance)

We will now define some functions and make use of the InceptionV3 model so that we gather the FID score of our model.

7.8. InceptionV3 Model for FID Score

We will now have to create the InceptionV3 Model so that we can get our FID score. Here, we will also prepare the test images to fit inside.

8.DCGAN Model Training

8.1. Train DCGAN Model

We will now start training our model after everything is done.

9.DCGAN Model Evaluation

9.1. Plot Generator and Discriminator Loss Graph

We will plot the generator and discriminator loss graph and observe both model's loss.

9.2. Make GIF of generated images

We will create a GIF so that we can witness the overall process of the DCGAN process.

9.3. FID Score (Frechet Inception Distance)

Here, we will calculate the Frechet Inception Distance (FID). The FID score will tell us the wasserstein-2 distance between multi-variate Gaussians fitted to data embedded into a feature space.

10.References

Using Google Colab More Efficiently

  1. Getting the Most Out of Your Google Colab (Tutorial)
  2. Using Google Colab with GitHub
  3. Cloning Github repository to Colab Notebook

Background Research

  1. Large-scale CelebFaces Attributes (CelebA) Dataset
  2. Pytorch Mnist Celeba Cgan Cdcgan
  3. How to Train a GAN? Tips and tricks to make GANs work

How to split original dataset into training, validation and testing datasets

  1. CelebA Attribute Prediction and Clustering with Keras

How to optimize GAN network?

  1. Tips for Training Stable Generative Adversarial Networks
  2. How to improve training accuracy of DCGAN (closed)
  3. Tips On Training Your GANs Faster and Achieve Better Results
  4. Understanding and optimizing GANs (Going back to first principles)
  5. How to Implement GAN Hacks in Keras to Train Stable Models

Key minor points to take note of

  1. Importance of buffer_size in shuffle()
  2. How to Identify and Diagnose GAN Failure Modes
  3. Curious about the constant used in the normalize function
  4. Clipping input data to the valid range for imshow with RGB data ((0, 1) for floats or (0..255) for integers)
  5. I'm trying to understand what BUFFER_SIZE in DCGAN does
  6. Epoch vs Batch Size vs Iterations
  7. Why do we write “ plt.imshow(generated_image[0, :, :, 0])” with a zero at the end?

Samples on how to code out a GAN network

  1. Generative Adversarial Networks GAN: A gentle introduction for beginners
  2. Advanced Section 8: Generative Adversarial Networks
  3. Developing a DCGAN Model in Tensorflow 2.0
  4. Tensorflow Deep Convolutional Generative Adversarial Network
  5. DCGANs (Deep Convolutional Generative Adversarial Networks)
  6. DCGAN — Playing With Faces & TensorFlow 2.0
  7. How To Build A Generative Adversarial Network (GAN) To Identify Deepfakes
  8. Generative Adversarial Network (GAN) for Dummies — A Step By Step Tutorial
  9. My first GAN using CelebA data
  10. Image Generation Using Keras Framework

Evaluation of DCGAN

  1. Sample of how to implement FID in DCGAN
  2. How to Implement the Frechet Inception Distance (FID) for Evaluating GANs
  3. How to Evaluate Generative Adversarial Networks